Universal Approximations of Invariant Maps by Neural Networks

We describe generalizations of the universal approximation theorem for neural networks to maps invariant or equivariant with respect linear representations groups. Our goal is establish network-like computational models that are both invariant/equivariant and provably complete in sense their ability approximate any continuous map. contribution three-fold. First, general case compact groups we propose a construction network using an intermediate polynomial layer. invoke classical theorems Hilbert Weyl justify simplify this construction; particular, explicit ansatz permutation-invariant maps. Second, consider translations prove several versions convolutional limit signals on euclidean spaces. Finally, 2D signal transformations group SE(2) rigid motions. In introduce “charge–conserving convnet”—a convnet-like model based decomposition feature space into isotypic SO(2). be approximator SE(2)—equivariant transformations.